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ABSTRACT^-' . . , ^ a...- 

. Thi,s paper presents a comprehensive framework for tti< 

evaluation of online bibliographic retrieval services and provides an 
example of its use/ with the Educational. Informaticn System for - 
Ontario (EISO)-!.^ A /systems apnro.ach, incorporating traditional ^ 
categovies of input, output, process, atid feedback^ is used to ^ 
develop the mcdeT''; in its application, particulax emphasis is placed 
upon" assessing 'relationships between variables cf interest at • 
different stages in order to measure EISO user satisfaction. 
Methodology is described, including the sociological variables used • 
to characterize EISO users; path diagrams and datei\ tables are 
provided; / and findings^,are discuss€dc (Author/JD) \ •. 
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A Path Amlysis Model Using Sociological 
Variables to Examine User Satisfaction 
\dth Qn-Line Education Information 
Systems 



Accompanying the widespread introduction .of on-line bibliographic * 
retrieval services has been a recognition of the need^or ^ their careful* 
evaluation. Several approaches have .been taken to this prbbleoi including 
price-donand analyses (Cooper and DeWath, 1977), organizational studies 
(Wax, 1976), data-base evaluations (Lancaster, 1969), and end-tisers 
(Brickley and Trohoski, 1974). However, because thes^studies have taken 
relatively narrow perspectives, they have failed to assess the inter- 
relationships of various components of the services in question.' In order 
to l^acilitate this type of analysis, a comprehensive frameivork for the 
evalViatiori of on-line uibliographic retrie-val. services- /is needed. The. 
purposes of/ this paper are to present one such framework, and to provide - 
an example of its use. * ^ 

A systems approach ivas used in developing the framewrk, incorpora- 
tin^, the traditional categories of input, process, /output ^ and feedback. 
In its. application, particular. CTipIiasis is placed upon assessing the 
relationships between variables of interest at different stages, such as 
-input and output, while co^rolling, for the effeqts of intervening 
variables. It is hoped -that in achieving a bett^ tmderstajiding of such 
relationships, those operating search services can make more rational 
deqisions about policies and procedures. 

SystQTis Framework for .Evaluation 

I ' In the evali:iation framewrk proposed here, a systems app:^oach is 
u^ed to describe the cycle of activities that occur ivhenever an on-line 
bibliographic retrieval service responds to a user's tequest. In doing 
p, four basic questions must be ansi^rered. IVhat inputs are required? 



Apait processes take place? \Vhat outputs are produced? IVhat feed^ck 
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- occurs at each stage? A. description of the sequence of events that 
occur .during a search qycle provides answer^ to these questions. ^ >i 

/ • The search c)^le begins ^w^^ a requestjby a user 

, to the search analyst, a specially traine<i reference librarian. They 
thenV"negotiate" the question, until it can' be restated in^ a form that 
. the search analyst can anploy to develop a search strategy; that caji be 
used-^to retrieve relevant citations from appropriate data-bases sixih as 
the ^Educational Resources Information Center .(ERIC) or the Ontario 
. Educational Research Information System CONTERIS) files. The analyst / ' 
/ ' then Queries the data-base Cs) using a retrievaL system such as LocWieed . 
Information Systan ,(IIS>*s DL^LCX^, System Development Corporation's. 
(SDC)*s ORBIT, or United, Nations Educational "^Scient if ic and Cultural 
Organization - (UNESCOys CDS/ISIS. When combinations of terms are 
entered, the number of relevant citations and a few sample citations 
are printed • If the citations are judged to be relevant to the client's 
question, the complete bibliography is printed offline and mailed to . 
the requestor . 

The user, on receipt of the 'bibliograpliy, reviews its contents and 
• makes a personal assessment of its^relevajice arid thoroughness. As a 
result, a request fbr original documents may-^be ma^e. 1Vhen these are , 
.delivered and read, the search cycle, as we define it, is complete, ^ ,/ 

' Feedback may occur at aljnost any stage of the search cycle. During 
question negotiation the\ search analyst>m§Lkes- inquiries' a^^ suggestions 



about /the search topic, and 'the user respond ii..,^^^ 
references, a discovery that an excessive number of^itations have been ; 
located, that citations are not relevant to the topic, or that other 
terms should be used, may cause negotiations to be reopened and the search 
strategy altered. Even at the end of the entire cycle, feedback occurs 
when an ijiformed and satisfied (or utiinformed and dissatisfied) user 
returns an evaluation questionnaire. 

/ 

/ This cycle of ::ci ivities forms the system portrayed in Figure^ 1. 
At each stage, three *^actors*' are involved: the user, the search analyst, 
and the -retrieval system. Each of the three is the referent for a number 
of variables \^ich describe the characteristics of the search service's 
ii5)ut!5, the nature of its processes, arid qualities of Its outputs. 
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This systolis framework -provides a tempofai sequence to events 
occurring during a search cycle ^vthereby alloidng the, evaluator to make 
" reasonable Jbaferences about cause and effect relationships This is 
important, since policy makers must have a reali^ti^— idea of t^ 
ultimate effects of different decisions about 'inputs and the organiza- 
tion o£ processing activities. In addition, the framework 'suggesta. a 
categorization scheme for variables according to their roles as measures 
of inputs, processes, or outputs. 

Application of Model ' 

The remainder of this paper describes the evaluation of one aspect 
of the Educational lri£gxmt±OTrSyste^ (Auster and La^./ton, 

.1976; Lai^rton and Auster, 1979) using the systems framework portrayed' in 
Figure 1. The pirqDOse of this application is 'to determine to vfliat 
e}ctent a client ^s personal and professional background is rela1:fed to 

V i - ■ " 

his or her satisfaction with EI SO. This assessment should pro\jade an 
indication o£^ those identifiable sub -populations of users . 'vjfeose needs 
are not currently being satisfied ars fully^a^^ of others. 

( ■ * Sample 
The population for this study is the ifniverse of all 371 searches 
requesfeSTfirom EISO between May 1, 1976 arid April 30, 1977. Though 
some data were available on all . searches , the present analysis was 
restricted to those which ha*d been run on ERIC and for which the 
requestor had returned an evaluation questi6nnaire. T!ie total" number / 
was 148 (40% of ail searches), but missing responses on some variables- 
result in a smaller number being used for some analyses. . 

Virtually all searches weye placed by users from the professional ^ 
education xomrrcanity in Ontario, i\/hich EISO ivas designed to serve. /Iji 
general, 40% of EISO' s users viere from school boards, 40% were from 
•OISE,10% from other universities arid faculties of , education, and 10% 
miscellaneous. Approximately 30% of the users reported thatt^ they. were 
engaged iji adminisnration, 17% in teaching, 12% in research^ 18% in 
graduate study, 7%/in librarianship, and 441 other. , . - 



' : x * * ; Methodology ^ V ^ \ / 

Path analysis was selected to ai^alyze the relationships among , \ , 
input, process /.aiid ou^ 'because it provides a method of ''\ 

testing.;the validity/of causal inferences for pairs of variables ivhile 
controliirig for the effects of other variables (Nie, 1975., pp. 383-397). 
. In addition, i^th di,agraias provide heuristic portrayals of systons 
?. relationships vJiich are well suited to tl^^' systans frameworlc used J:6 
organize^the variables in tni^^ study. 

ITie analytic technique used in this investigation follows the 
practices advocated by Goldberger (1970)7" In pafticular, the path 
coefficients presented here are the standardized regression coefficients . 
resulting from regression of the dependent variable in question on those 
variables directly affecting it/ and those alone. In c^ses whe^-e there 
is but one predecessor variable, the patlj coefficient is identical to 
^ the . zero -orddjr correlation coefficient; in other cases it is equivalent 
to the partial correlation coefficient of the . dependent variable on the' 
standardized predecessor variable contrplling for other variables 
directly affecting' the dependent variable. , v . ' 

The amount of une[3q)laine^.^ra^^ in any dependent variable 
.is assumed to be accpunt'ed for' by hypothetical "erroir" variable s denoted 
by e.. The. correlation bet\\reen e*. and Xi . is equal to Jl - where 
is the multiple correlation beti\reeh X.;. and all the . predecessor variables 
directly affecting it. 

: ^ Not all reXationshi|)s for which no causal inferences are plauTsible 
^ may be omitted. ' Second, relationships between exoc^enous variables 
(portrayed at the left bf the path diagram) are t)T)icially unanalyzed, 
though they are sometimes controlled for if they are considered to 
interact mth one another. Finally, correlations among the e^ are' 
/ksisuined tb'be zero and romain unanalyzed.\ ' 

. Path analysis requires three principal assumptions which may or may ^ 
not be valid. 'Fir^t, all relationships are assumed to be linear ;v second, 
relationships ai^ assumed to.vbe additive; and third, vit is assumed no 
interactions (i.e., multiplicative effects) exist -aniong variables. As 
well, some authors^ would hold that the variables are assumed to be/mea- 
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sQted on a ra:ti6 -^^P^ '.-ath some 'fixed iinit of ^easure. 



/ 



■^^'■^br .the dichotxjmous variables, such as sex, and those measured in 
standa!i5d units o£ measu^ such as the ' turn-around timey none of the 
assi#ptions^ TOuld bfe violated. Those measured on Likert scales, such as^. 

Jsatisfactipn, may violate assunptions as to linearity and unitary^ ^ ^- . 

>measurenient.. However, because monotonic, if not linear, relations &re . 

^(Kcpected for1a^^^ relatioijships dprolving thfesev:trariables, vioLatiori pf . 

.•the first as(5umption wduld, .at viorst, cause the relationships^ to "be • P 

iunderestimated. . ' . • \ 



Colle'ctidn of Data 



V major isources of data used l;Ln these, evaluations were a User 
E\ralu4fio%Q^^^ arid a Data Sheet' co)npleted . by tte ^earch aiml^^ • 

In all, 131 different variables were regularly collected for each search 
that was completed; Selection of 'these items ms guided by key concepts . ; 
in^the disciplines of sociology, psychology and econorflics. Most ^vere dram* 
fi-om the literature of previous evaluation studies but only those< that were 
defensible measures ^of.^rc53;es , status, laiowledge, opinion, post, price or : 
demand were included. TVhere gaps appeared, additional items were developed. 



V Measurement of Variables ^. 

^ Descriptions of thfe items used to measure tiie variables included, in 
the analysis ^which focuses on the influence of the. sociological .back- 
'grouni of clients on tiaeir level of sat^ ^fact-r/n are listed in. Table, 1. 

The first five variables in Table 1 arfe^ input variables related to 
the backgrounds of clients. All .are dichotomies coded 1 oi' 0 . > LOCATION" 
refers to the location of^-^^ie^ organi:!ation' in which the usei^ i>ra.s ^ployed 
or enrolled as a student. SEX^needs no explanation. ' ORGANIZN is the 
organization in which the^^user ivas employsd or enrolled as a student. . 
The PURPOSE of the search refers, to the use made of the search, whether 
.it.^vas for professional or acadsTiic purposes: PROFROLE indicates the ' 
user was- in an .administrative or non-administra'tive professional position. 
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■ ; .' .'TABLE 1 : I ■ ■ , ■ 

VARIABIES USED. IN- EVALUATION OF USER'S BACKGROUND - 
■ ,. ■; AND LEVEL OF SATISFACTION 
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•Variable*^ 



Number 



Name 



'Item Description and Coding 
. Description 



Coding 



7 

9 
10 

11 

12 



LOCATION 



SEX 



^ORGANIZN 



PURPOSE 



PROFROLE. 



SYSTEM 



SATIS6' 
SATISl 

SATIS2 

SATIS3 

s 

Y .SATIS4/ 

\ ' 

SATISS 



\ 



Metropolitan Torontb! 
Oufside Metro Toronto 

Female 
Male 

Student, or staff of OISE 
Non-OISE, 

Purpose of search: 
^ Non- academic 
Academic 

Professional role: 
Administrative 
„ Non- administrative 

Search system lised: 

SDC OI^IT 
fLIS"' DIALOG 1 

"Overall satisfiSction " 

Satisfaction • witi-, /publicity 
- materials and directions 



Satisfaction with convenience \ 

: '-and''heli)f ulness j ^ . ' 

Satisfaction with timeliness 
of service • . ^ - 

Satisfaction with quality of | 
techno^logy ^ y 

Satisfaction with, value of 
bibliography ^ 



1 
0 

1 
0 

1 

0 • 

t 

0 



1 

0 




^Variables } through 4 and 6 are recorded by search ai^lyst on data 
. sheets. ; Variables 5 and: 7 through 12 indicated by client on User Evalua- 
tion Questionnaire. / . N 

f. Table 2 ; for complete list of satisfaction y^ems and construction 
* :-of suBscales.c .\ v 
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Tlie final input variable, SYSTFM, refers to/ the computer search ' J 
;,service ,4at was used in executuig. the search :/ sE<: 0FlB-I3:_pr^LIS Dm^^ 
Although this variable i§.,a system rather than user input, it v.-as inclu- 
-ded in/tJle aiialysis'l^^^ differences bet\'ieen the b^ro; influenced 'the 
selection of the system used jn conduct&g searches. . In particular the 
direct mail' service provided by SDC, which allows the computer -prodiiced 
bibliography to be .sent directly .fo the. client from SDG , meant tKat this • 
service might be. V preferred for users located some dist;ance from Tor6n1:o. 

. . User ktisf^ction with various aspects of the. ^ea^icji service were 
measured -by fiv-e ^a&ales^ CS/^TISL to SATIS5) consistinjg' of scales tha;t' 
iiicluded satisfaction with publicity materials and directions : (SATISn.')-, 
with convenience and helpfulness (SAnS2;,wit3i_timL>linfe^ of .service 
(SA.TIS3) , Vith -the' ^n'ty of the; bibliogi^aphy, CS/^TIS^\) . , y ^..^ 
^--'Individual items used iii constructing, subscales are report,ed in. 
<Tabie_2 irid include those used in the: scale lis ed to measure overall , 
"satisfaction (SATIS6) . The latter scale u§ed five items selected from 
'the subscales; not all items were iiicldded bec' i-^e mpjiy cl;Lents had not 
been exposed to all aspects of the^service an4 therefore/had failed to • 
respond to some°itans. Instead, only those items referripg to matters 
to ivhich all clients had b'een exposed wcr'^ included in the scale. 



Reliability coefficients for I3ie Mx' scales Wre .80, .62, .7^i .2T, 
.'79,„and .80 respectively. Overall, these coefficients are sufficiently 
iWh for the scales to be used in assessing the satisfaction of/ EISO's 



lisers vath thei ^s^ervice. 

The fact that all input variables .are dichotomies call? for specici . 
care i.-'/int'^reting correlation or path coefficients between, these and 
bther variables i ,' For example,' a positive correlation b^ween location 
and organization would dimply that use-.-s from. Metro; Torbnto (coded.l) tend 
to. be associated with OISE (also coded 1), whereas those^fr^outslde ., 
Toronto (coded 0) tefil to be .associated mth otheiv^nst-il^u^ G^lso 
coded 0). TVhere the second variable is continueils^ as in the case of 
-satisfaction, a. positive correlation Jjjiplles. high satisfaction is associa- 
ted with the tvait code<^l mth the^^tralt coded 0. . , 
In the'case of rel^iria^Sip bet^>reen, say, loc^-ion and satisfaction,-,' 
a positive coi'tv .di^rould imply that users from Me tiro Toronto (coded ... 
1) were more satisfied than those from .outside Toronto 1 (coded 0). 
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: V: . • . TABLE 2 >; . . : 

ITEMS USED IN SIX .SATISi^ACTION' SUBSCALES 



•A" 



Question: Please indicate your satisfaction with tihe; fallowing ^elements . 
' Of ybuy EISO search. 



SATISl - Pubiicit)^Matkxiar^and'D.i^^^ . \ 

' * s a: ' Convenience of arrangements and adequacy o£ diractioiis 
b. Accuracy and comprehensiveness of publicity materials 
^ • • • c. Adequacy of- directions for ordering inaterjals" 

SATIS2 Convenience ^ar>.<^ Helpfulness \ ' 

a. Convenience of- arrangements and adequacy of ;direction 
b; HelpfulnesiD of 'search analyst ; ' ^ - 



SATIS3 - , Tijneiiness of Seindce. ^. ; 

a. Tme taken to deliver "bibliography 

b. Time taken to deliver materials frpm^EE^RS 
^ / a. Time" taken tio deliver materials from EISO 



SATI^4 " Quality, of Tecmolog)^ ^ . ^ 
/ 2l,1- length of pibliography 

/ . ' ,^b. Readability of microfiche copies 



L K 



V c. AvailaJ)iyLty qf microfiche readers' 

SATIS5 - Value of 'Bibliography and Materials 

a. Bibliography itself • / • 

^ b. . Materials located via bibliography 

SATIS6 - Overall Satisfaction 
la, 2b, 3ci, 4a, 5a 



Note: Scale scores ^vere adjusted^r the niM»er^of items by dividipig 
total scale, scores by the fvLiinber of items. Scale: tow =%1; 
f medium == .2 ;\ high = 3» 



^^:.\ ^ Sociological Inpuu.Mc^^ , \ : 

. Who ;are Eis6»s users ^od ikn^ does tijeir identity affect- 1^^ • 
of satisfac^^^^ the service? As suggested above, thels^ are m 

i^estions ,>aiic3^ i^iefl.ect tlie success^ of 'the servic6"^in. acMev^^ 
By 'implicatip^^ one* kind of client is |less, satisfied than anotlier/ * . 

^changes s!^icfeomight be nep^ssary in-order to raise their level ^ • ; 

of -'Satisfaction^;; \. j • ... // ' ■ '-^ ' ■ 

• The question as to the. iisers* Identities is (answered here in terms^ 
of the' tv\ro. !5pciolo^ica^ states We view 

these as/characteristic^ that clients bring vali^^ thesiV to their eiicoim^^ 
with the search s^icTe, and as such represent irqputs^to the.syste^ 
/status is itself pn large part an attribute associate^ 

it is' difficult t^b measure "these, tvro '^aspects^sejparately.N ^^^1^ of the v : 
•va:riables we select(sd reflect, to- oife^djegre^Nnr ano-dieri;:/^ - 
role and status /of the individual client'^ > ' v i 

, \\ . \ . . ./•-'■■K/' ^;v/' "■ ^ 

. Description of /^odel' V ■ \^ \ - ' /■v./x v ;"^/,:,/;^^^;••^.■^■ _ 

/| • • •The image guiding the selection of sociological^variables to charac- : 
t.efizet:.IS0V3;'Users ms, that of a professional eauca:tor workitig/or 
studying at r some location in Ontario. Tlie user* s/ geographic location ^ ^ . 
(lOQATION) CSEK). were viewed as tvvo Jbaclcground viariables. of 

possible importance. Location is a factor of considerable policy ^irnp^"' 

stance since one 6^ EISQ*s major purposes iis to malte the resources^of a/^; 

;major\ res^*irch^Alibrai^ available to educators throughout theVpr^ince . ' . 
To be sure,^rrent^lirni^tations &f automated retrieval systems make this 
a distant goal, yet the quiclc ret^aeval of both references and original 
documents in R'lE, CIJE>^and ONTHlISvd^^ provide access to a;large and . 
in^ortant body of .laipwl edge.; IVhile sex of a client is notXin and of > 
itself, of primaiT- in^ it is a factoi;; known t6v be related': to the \, 

role of professional educators; in particular, relativaLy f ew wmen / 
currently hold. admirtistfative positions i^ithih Ontario *s^educational'. 
system: Thus, sex i^s viewe d as- a backjgr bimrl variable-...that-^- should.be 

. ■ ' ' • - ■ ,■ ' ■ '"'i 

contix)lled.--' / ^ - ti ■ \ \ ,. v - 

A client's organization (pRGANI2^^^ 
and purpose in ' conducting/ the s^ (TURPOSE)\were viewed as th^. major ^ 



"characteristics defiiiing his or her role. Though the coding scheme 
•use4 in "this amiys is. greatly simplifies the original categorization 
scheme> Sre originally determined whetiier or not a user in any one 
o£ 29 different types of ^ganizations or units; e.g., prescliool, 
public school board, separate school board, private, school^, College of 
Applied Arts and Teclmology, faailty of education, etc. Professional 
roles were classified into 14 categories; e.g., administration or 
supervision, teaching, piipil per some l^jserv-ices, research, etc. 
Finally, the purpose of .the search, which we saw as. au objective in 
large part determined by the individual's organization^and rqlei was 
initially pl^aced In one of eleven categories; e.g., keeping abreast of 
the field, class assignment, preparation of a bibliography, curriculum 
development, etc. 

To illustrate the connection among HiQEROLE, ORGWIZN; and HJRPOSR, 
consider tivo typical users; One might be a full-tiine graduate student ^ 
at a faculty o;f education conductiing a literature search for a class 
assigment; the other might be a senior administrator in a public 
school board seeking information ;;about curriculum guidelines. 

Anothier input variable related to the service's activitieis is the^ 
partlciiLar search system used (SDC^ or LIS's). tVhile not a sociological 
variable, this characteristic (SYSTBI) is included as a factor of _ L 
possible iinportance because, one of the s)^stOTS SDC, offers the ^c^^ 
of mailing bibliographic output directly to the cliBnt.' Foythose clients 
in ronote locations, therefore, SDC may~TDe a preferred service in terms 
of * their satisfaction ivith the service's ^turnaround time. ' \ 

/Finally, the output variable of interest is satisfaction^^ which ^vas 
measured by 12 itoms relative to five distinct aspects of 'the service: 
publicity materials and directions (SMISl), convenience and helpfulness 
(S\TIS2), ^ timeliness of service (S\TIS3), quality of technology CS/VTIS4), 
and value of the bibliography (SMISS) ; A subscale of six itms ^>ras also 
derived to measure overall satisf^tion CSATIS6>. 

' Organizing the variables described above into a formal path model 
results in the diagram pictured in Figui^i- 2. LOCATION and SEC are back- 
ground variables whose relationship to one'ai^ther renjains unanalyzed,- 
but whose effects on other variables are consid^^. Location may affect 



"satisfaction both directly and indirectly. • One indirec^-pat-h- is through 
its imluence on the systein used. ITie second effect is via its influence 
on. the type of organization from ivhich the client comes. Sex, too, may^ . 
affect satisfaction both directly and indirectly, ^e indirect effect^-- 
may be through th e_type of organisiation or the individual's professional 
role. ^ 

Organization stands as a key intervening variable betwe^i the back- a- 
ground variables and satisfaction, on which it may liave a. direct effect. 
However, it may also have indirect effects via. the individual's profes- 
sional role and the purpose of the^search. Similarly, professional role 
may affect satisfaction directly or through the purpose of the search, 
while the latter, variable^s-^postulated to have only ^direct effects. 

' It may be noted that' other paths than those noted are possible; 
vhowever, we are including only those which seem logical given knoivn 
relationships and^ t^e meaning of the variable^. Thusv-^f or example', we 
haveT-omitted path^/'connecting, say'SEX and SYSTEM. It should also be 
noted that where direct effects are suggested, these effects are residual 
to the effects through other variables. Thus, the path connecting 
organization and satisfaction represents the effect on satisfaction of 
the ojidividual ' s brganizatiom after the effects of the person's 

professional^. role and purpose in conducting the~^search have been removed. 
All rj3sidual variation wMch is not accounted for by any of the variables^ 
in the model is assumed to be caused by external variables, represented^ ^ 
by the e^'s. - . 

The problem for this particulai^ analysis, then, is to. test the 
validity of the inferences about cause and effect relationships sugges- 
ted by the path analysis model -depicted in .Figure 2.. IVhile confirmation 
of the relationships described by the model does not prove the existence 
of the iJTiplied causal, relationships it ivould provide evidence supporting 
their plausibility. . ' / 



Findiiiigs 



Jn this section, we shall proceed from the general to the specific, 
presenting first the findings for the path analysis in Which overall 
satisfaction (SATIS6) is the dependent or output variable, and then the 
resul.ts, for the five satisfaction subscales. Table 3 lists the correla- 
tion coefficients, means, and standard deviations for the seven variables 
that appear in the path diagram in Figure -2. The means for the sociolo- 
gical background variables need some interpretation since the variable's' ' ' 
themselves are dichotomies. i - 



r Stable .3 

CORRELATION MATRIX, MEANS, AND STANDARD DEVIATIONS 
FOR VARIABLES IN SOCIOLOGICAL INPUT 'MODEL- EXPLAINING 
OVERALL SATISFACTION WITH SERVICE 



^^2 



Xi LOCATION 



.X2 SEX ^ 



.095 



X3 ORa(NIZN .511"^ .087 



X4 PURPOSE -.430' -.197^ -.434' 



Xs PROFROLE -.135^ -.309 



.348^ .294"' 



Xs SYSTEM 



.093 ' .004 .007 -.059 .030 



X? SAtlS6 



-.050 



.024 .053 '.035 '-.081 



.094 .. 



Meaii 

Standard 
.Deviation 



.531" .364 . 352 . 469 . 327 ^154 2.S^ 
.501 .483 .479 .501 .471 .362 0.37i 



N = 162 



for IXX^ATION indicates that 53% of the users in the, 
sample were from Metro Toronto and/ conversely, tliat 47% were from out- 
side Toronto. Similarly, the means foi* SEC, ORGANIZN, PURPOSE, PROFROLfi, 
and SYSTpi can be interpreted, in terms o£ percentages: 361 were students 



or stai^f at OISE: 47% of all searches were for professional as opposed ti^ 
acad^ic purposes; and 15% of all searches were conducted using SDG. Th^^ 
liieaji of 2»59 (on a scale from 1 to 3) for overall satisfaction indicates 
the average user i^/as highly satisfied. - 

The zero-order correlaticij'iS among the five sociqlogical bacVground' 
iri^bles are seenj for the most' part, to be statistically significant. 
In contraist, neither' SYSTB1 nor S/VTIS6, overall satisfaction, have any \ 
statistically significant relationships with^the other variables included 
in\the" an^^sis.- 

The path analysis in Figure 2 shows that the pattern of correlations 
exliibited in the correlation matrix are present even when 111- of the 
backgroundwariables are entered, • ^ , 

IjCCATI(^has a vety weak, non-si^ . 
SYSTBl", indical^g that SDC ivas not- used more often for" clients distant 
from Toronto. Given the coding -systm, a negative relationship touIcI 
have been indicated. Location -has a non-significant negative correlation 
with satisfaction. WiB^t slight relationship there is TOuld suggest 
grfeater satisfaction among , those users outside -the immediate metropolitan 
area where EISO is located. Finally, location does have a strong positive 
correlation i>rith organization, confirming "that most of OISH*s staff "and 
students who use the system are associated >d.th the 'main campus in ' 
Toronto rather 'than one of its nij>e field centres. 

^ ' ■ . - . •' ■ A 

' . . ■ N- • . ■ ^ ■ - • 

• \ , ■ • ■ 

SEC, the other prior variabre contained in 'the: path model in Figure 
2 exhibits' only one significant relationship, that' with PROFROLE. t\This^.^ • 
relationship in5)lies that JEISO's wmen clients tend to hdld non-administra 
tive positions', and, that its male clients tend' to hold administrative 
positions.; ' , . , \ \ .\ . 

At ; the next stage. af analysis, ORGANIZN is seen to h'ave significant . 
relationships with both the purj^e of the search (the negative path . . . 
coeffidient implies thoseSDutside OISE are more> likely to t'equest searcTies 
for professional rather tlian acadonic purposes) -and the prof essi 
of the iixiividual (the negative relationship implies that OISE users, tend; 



TABLE -4 



,. FOR SATISFACTION ifllT-I SERVICE 



Path 



/ 



1.. LOCATION-SEX- 
; 2Ji ORklZNOTION, •■ 
■3, 1 ORGANIZN-SEX > 

■ 4; IPURPOSE-ORGANIZN * 
S. lPURPOSE-PROFROLE J 

_ 6. PROFROLE-SEX' 
7, PROFRDLE-ORGANIZN 

■ ■ 8. . SYSTEM-LOCATION 
9/,SATIS-LOCATI0N 

10. SATM 

11. ■ SATIS-'ORGANIZN 

12. SATIS-PURPOSE 

13. iSATIS-PROFROLB 
" W.' SATIS-SYSTEM 

" s . 



Statistic 



Variable 
ji — _ 



SATISl. .mm , ; SATIS3 : /:SATIS4 ; SATIS5 , SATIS6 



.11 

..52' 
.05 

■,36\ 
.11- 
-.32^ 
-.31+ 
.06 
.11' 
.08 
.03 
.18^ 
-.08 
.01 ' 



;16^ 
•..55''* 

' -.39^' 

.15^5 

' -.28"^ 
-.34+ 

\M'. 
,. .11 
•-,05 
- .11 
' .13 

■ .11 



171 
2.57 

i46. 



-.06 , 
■ .25' 



.11 

JO^ 



-.01 -:04 . ' M 
■.36^' -.33 
■.07; \' :.275 



;-.42^ 

-.39+ 

.18' 
\37 
-.21' 

.06' 

.19 

;^3i . 

- .07: 



-.35+. 
^.33+: 

;,.o7 

; .02 
: -.07 

\l2 



' 182 V 32 i 75 
■ 2.77^, ■2.41/t.;2,Sl 
.37/ 'f ' .\^3. ' : v,38 



.34 



t 



V.l5;^ 

-.31+ 

%30+ 



-.23 
.02 



1 



.03 

s09 



148 



:.09 : 

;.5i^^ 

■.04-^ 
-.3p, 

.16 
-.28+ 



-.09 
■.04 ' 

".10 . 

.06 

-.09 

■.11': 
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2.32>-'':' 2.59 

:':^;::''"..34 



V. 



not ;to be ,adnd^ , lyhile profess ional clients tend to .hold' 

adndnistrativ^ jtositions) /< Finally, ORGANIZN has a residual positive 
effect, on satisfaction, after the ranoval of effects due to SEC, 
LOCATION^. IWP<)S 



-1f 



/ • The last stage of the piith*. diagx^am involves . SYSTEM; PURPOSE, 
an^vTTipFROIJE/: S^^ seen to kave no statistically significant , 

\^el^ect bn satisfadtion. Similar conclusioriS hold foj; PURPOSE --^ ;| ' 
kcadonic and ndn-acad^erfic users are ecj[ually satisfied - - i and PROFROLE 
' administrators and non-admiidstrators ^re equally saliisfiect. 

, V TuiTiing now to the path analyses for /the satisfaction subscale 
reported in Table 4, we note tliat_that pattern of p^th coefficients is . 
veiy similar to that observ^ed for overall satisfaction. ..Jn fact, all 
Coefficients not involving; a satisfaction subscale are in fact 
estimates of the same^coeff icients repoVted under the preceding ^analysis . 
■ They differ only /because of the different sizes of the' 'subsamples for 



wJiich complete data.v:^ire available.^ 



/ 



Concentrating on the satisfactiou subscales, we note two signifi- 
cant correlations. Since, amqn^ the 36 path coefficients, wuld 
expect a^)out this' number to be/statistically -significant a.t the .0.5 -level 
by chajice alone even when there were no real r.(Blationships, we should 
not placejf much weight upon them. The t\yo paths we refey to are between 
PUrUPOSE and SATISl,., satisfactiqn with publicity materials and directions. 
.This relationship would imply 'slightly greater satisfaction iviith these 
among users requesting searches for non^academic puiposes.-- The second ; 
significant relationship^ between LOCATION, and \^TIS5,7 satisfaction mth 
the value .of the bibliography ancj materials, is'^egative. This ivould 
imply that users from outs i9,(5 a^ more satisfied with the end result of 
searches than are/those from inside Toronto. 




ERIC 



Concitision'- °' v • - ' . •. \l 

. ' major markets idth zqual effectiveness. One of tljese, markets, the 

■ profei^sioiial marlcfef, is composed espeCikLly of male administrator-^ v 
•. ; - located throughout- Ontario . The ether, academic market, is composed-^ 

• ; . . of student^^^^^^^^^ 

vromen than^^^u^^ other, and i% concentrated in Toronto, especially at 

X \ ; ■ i '.- 

" different levels of client sat.isfaction can be taken as mark of EISO's ,, 
siicces^ Housed as it is in an academic library at OISE in Toronto, 
r - •■ - one i^ould expect a tendency for it to serve- its inmediate constituency^- - 
'\ ^best. ' The very lack of such a relationship indicates it has been 
\ successful in satisfying the needs of professional educators \yxth equal 
'\ ef ifectiveness . . 
, ' \ • It should be emphasized that EISO i? effective. The mean score 

^'1^ ■ % \ / for the clients* 0^^^ satisfaction was 2.6 on a three-point scale^ 

'■jr/:\ The subscale m eans in Table 4 show that u^ers are also highl)^sa€isf ied ' 
\ ^ \ idth EISO* s publicity mterials' and directions (a\TISl)^JL'ts convenience 

.' ;\and'help&lness (SATIS2)7-andi:0^^ 

■ ■ Vrhe users reported \ moderately high level of satisfaction on the^ ■' 

r tentaiiiing two subscales, timeliness of seprice (§/VTIS^^^ 

r '• ° of the bibliograiihy ai>d materials (SA.TIS5>. Given that, the timeliness 

service was in part dependait on delivery of ii^terials from EDRS in 
; . U.S., we can even discount ,\ to some extent, one of these two. , 

r:,r " ■ ,, The fact that the sociological input model has failed to explain 

:\ . ^ ^ different lev^l^ of satisfaction does not mean^the small variation 

. bbserv-fed in, these^ variables^ camot be explained; t^^ 



models that artelnore effective at this. task. 
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